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CHAOTIC BOLTZMANN MACHINES WITH TWO ELEMENTS 


HIDEYUKI SUZUKI 


Abstract. In this brief note, we show that chaotic Boltzmann machines 
truly yield samples from the probabilistic distribution of the correspond¬ 
ing Boltzmann machines if they are composed of only two elements. 
This note is an English translation (with slight modifications) of the arti¬ 
cle originally written in Japanese [H. Suzuki, Seisan Kenkyu 66 (2014), 
315-316], 


1. Introduction 

Recently, chaotic Boltzmann machines flU] were proposed as a determin¬ 
istic implementation of Boltzmann machines. The apparently stochastic 
behavior of chaotic Boltzmann machines is achieved, without any use of 
random numbers, by chaotic dynamics that emerges from pseudo-billiard 
dynamics. It was shown numerically that the chaotic billiard dynamics of 
chaotic Boltzmann machines can be used for generating sample sequences 
from the probabilistic distribution of Boltzmann machines, and it was suc¬ 
cessfully applied to other spin models such as the Ising model and the Potts 
model m 

Despite these numerical evidences, there have been no theoretical proof 
that chaotic Boltzmann machines yield samples from the probabilistic dis¬ 
tribution of the corresponding Boltzmann machines. 

In this brief note, as a first step of theoretical approach, we investigate 
the simplest system. Namely, we show that chaotic Boltzmann machines 
truly yield samples for the corresponding Boltzmann machines if they are 
composed of only two elements. Although our approach cannot be applied 
to larger chaotic Boltzmann machines with more than two elements, we 
expect that it gives some insights into the dynamics of larger chaotic Boltz¬ 
mann machines. 

Since the proof is not entirely trivial, it is considered worth making avail¬ 
able on arXiv. This note is an English translation (with slight modifications) 
of the article OJ originally written in Japanese. 
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2. Chaotic Boltzmann Machines with Two Elements 

Let S i and S 2 be random variables that take values on {0,1}. We assume 
that the probabilistic model P\S ,, S 2 ] is given in the form of conditional 
probabilities P[S 1 | S 2 \ and P[S 2 | Si], which is common in Boltzmann 
machines and many other spin models in statistical physics. 

Let us consider a chaotic Boltzmann machine for the probabilistic model. 
In addition to the states s\ and s 2 e {0,1} of the elements, we introduce 
internal states Xi and x 2 e [0,1] of the elements. Therefore, the state space 
of the chaotic Boltzmann machine is {0,1 } 2 x [0, l] 2 . The internal states X\ 
and x 2 evolve according to the following differential equations: 
dx 

(1) —^ = (1-2s 1 )C 1 (s 2 )P[s 1 l^r 1 , 
dt 

(2) ^l = (l-2s 2 )C 2 (s l )P[s 2 \s l T\ 
dt 

where C,(-) > 0 represents arbitrary positive constants defined for each state 
of the other element. When the internal state x t reaches 0 or 1, the state s, 
of the element changes as follows: 

(3) Si < — 1 when x t = 1, 

(4) Si *— 0 when x t = 0. 

The internal state {x\,x 2 ) moves straight inside the unit square [0, l] 2 , and 
changes its direction when it hits a side of the square. Therefore, this system 
can be understood as a pseudo-billiard in the billiard table [0, l] 2 . 

What we expect here is that the state variables ,V| and s 2 observed from 
this system follow the distribution P[S 1 , .S' 2 ]. Namely, our goal in this note 
is to show the following proposition. 

Proposition 1. For any initial values (.v,(0), .yffO), Vi(0), .u(0j) 6 {0, l} 2 x 
[0, l] 2 , the chaotic Boltzmann machine defined by Eqs. (IT]) — © satisfies 

(5) lim ]- f tf(s 1 (f),si)<5(j 2 (0.4)* = - p [' s i»' s 2 ] 

T ^°° l Jo 

for (,v |, s' 2 ) G {0, l} 2 , provided that ^^ is irrational. Here, 6(-, •) 

Rm+Rji) 

is Kronecker’s delta function, and RfsJ = CV/Csvj/ff.v,]. 

3. Proof: Quasi-Periodic Dynamics 

To show this proposition, we rewrite Eqs. (Q]) and © of the chaotic Boltz¬ 
mann machine. Specifically, instead of the state (s h xj of each element, we 
introduce a new state variable y, = Hfsj, xj using the map 

(6) Hfsi, xj = (1 - 2 sJRfsJxi. 
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Figure 1 . The state space for (y i, v 2 ). 


Then the state space {0, l} 2 x [0, l] 2 of the chaotic Boltzmann machine is 
mapped to the rectangular [—i(1), (0)] x [-,/? 2 (l),,/? 2 (0)] (see Fig.Q}. On 
this state space, the dynamics of the chaotic Boltzmann machine (Eqs. dT])— 
©) is rewritten as follows: 


(7) 

1 

ii 

^l(5l)Ci(5 2 )P[^i|^ 2 ] 

( 8 ) 

ii 

^2(^2)C 2 (5l)P[^ 2 |^l] 

and 



(9) 

)’i * 

-^i(l) when 

( 10 ) 

y« 4 

-7? 2 (1) when 


-l 


POi,s 2 ] 

Ri(si)R 2 (s 2 ) 

P[^l,5 2 ] 


y* = R i(0), 
y« = R 2 (0), 


where the state .v, can be determined from >’/ as .v,(y,) = 0 when >7 > 0 , and 


sfyd = 1 when y t < 0. 

The differential equations d7]) and d8]) always satisfy 


dy i 
dt 


Hence, 

dt 


the state (yi,y 2 ) moves in the direction of (1,1) in the rectangular state 
space. Here we introduce (z\, zi) which moves along the orbit of (yi,y 2 ) at a 

constant velocity = ] Then the orbit of (zi,Zi) is equidistributed 

dt dt 


in the state space in the following sense. Let us consider the Poincare sec¬ 
tion on z\ = -./^(l). While the value of Zi changes by (R\(0) + /?i(l)), the 
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value of Z 2 changes by the same amount. Therefore, the Poincare map on 
z,\ = -R i(l) is a rigid rotation with rotation number (i? 2 (0) +R 2 (l)) / (R\(0) + 
7?i(l)) mod 1. If the rotation number is irrational, the Poincare map is er- 
godic with respect to the uniform distribution. Then the orbit is also equidis- 
tributed in the state space, and we have 



(ID 


Since (yi, >’ 2 ) moves along the same orbit as (z .\, z 2 ) with the velocity de¬ 
scribed in Eqs. © and ®, we have 



( 12 ) 


which completes the proof. 


4. Concluding Remarks 


In this brief note, we have shown that chaotic Boltzmann machines with 
two elements have quasi-periodic dynamics and yield samples from the 
probabilistic distribution of the corresponding Boltzmann machines, pro¬ 
vided that the rotation number is irrational. 

The set of parameter values C,(-) that make rotation numbers rational has 
Lebesgue measure zero. This is analogous to probabilistic Monte Carlo 
sampling, which does not work with probability (measure) zero. However, 
when we design chaotic Boltzmann machines artificially, it may be possible 
that the rotation number easily becomes rational. In such a case, we have 
to adjust C,(-) by multiplying some constants to make the rotation number 
irrational. 

The performance as an sampling algorithm depends on the characteris¬ 
tics of the rotation number. At least, a chaotic Boltzmann machine with a 
good rotation number that yields a low-discrepancy sequence is expected to 
exhibit better performance than the corresponding Boltzmann machine. 

Our approach in this note cannot be applied to larger chaotic Boltzmann 
machines with more than two elements, which exhibit chaotic behavior. 
However, some aspects are expected to be shared with larger systems; for 
example, our preliminary numerical study shows that irrationality is impor¬ 
tant, for chaotic Boltzmann machines composed of not many but more than 
two elements, to generate faithful sample sequences. Therefore, we expect 
that our approach gives some insights into the dynamics of larger chaotic 
Boltzmann machines. 
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